Adaptation to motor-visual and motor-auditory temporal lags transfer across modalities

Previous research has shown that the timing of a sensor-motor event is recalibrated after a brief exposure to a delayed feedback of a voluntary action (Stetson et al. 2006). Here, we examined whether it is the sensory or motor event that is shifted in time. We compared lag adaption for action-feedback in visuo-motor pairs and audio-motor pairs using an adaptation-test paradigm. Participants were exposed to a constant lag (50 or 150 ms) between their voluntary action (finger tap) and its sensory feedback (flash or tone pip) during an adaptation period (~3 min). Immediately after that, they performed a temporal order judgment (TOJ) task about the tap-feedback test stimulus pairings. The modality of the feedback stimulus was either the same as the adapted one (within-modal) or different (cross-modal). The results showed that the point of subjective simultaneity (PSS) was uniformly shifted in the direction of the exposed lag within and across modalities (motor-visual, motor-auditory). This suggests that the TRE of sensor-motor events is mainly caused by a shift in the motor component

Selective adaptation and recalibration of auditory speech by lipread information: Dissipation

Abstract Recently, we have shown that lipread speech can recalibrate auditory speech identification when there is a conflict between the auditory and visual information (Bertelson, P., Vroomen, J., De Gelder, B, 2003. Visual recalibration of auditory speech identification: a McGurk aftereffect. Psychol. Sci. 14 (2003) 592-597). When an ambiguous sound intermediate between /aba/ and /ada/ was dubbed onto a face articulating /aba/ (or /ada/), the proportion of responses consistent with the visual stimulus increased in subsequent unimodal auditory sound identification trials, revealing recalibration. In contrast, when an unambiguous /aba/ or /ada/ sound was dubbed onto the face (with no conflict between vision and audition), the proportion of responses decreased, revealing selective adaptation. In the present study we show that recalibration and selective adaptation not only differ in the direction of their aftereffects, but also that they dissipate at different rates, confirming that the effects are caused by different mechanisms

No effect of synesthetic congruency on temporal ventriloquism

A sound presented in temporal proximity to a light can alter the perceived temporal occurrence of that light (temporal ventriloquism). Recent studies have suggested that pitch–size synesthetic congruency (i.e., a natural association between the relative pitch of a sound and the relative size of a visual stimulus) might affect this phenomenon. To reexamine this, participants made temporal order judgements about small- and large-sized visual stimuli while high- or low-pitched tones were presented before the first and after the second light. We replicated a previous study showing that, at large sound–light intervals, sensitivity for visual temporal order was better for synesthetically congruent than for incongruent pairs. However, this congruency effect could not be attributed to temporal ventriloquism, since it disappeared at short sound–light intervals, if compared with a synchronous audiovisual baseline condition that excluded response biases. In addition, synesthetic congruency did not affect temporal ventriloquism even if participants were made explicitly aware of congruency before testing. Our results thus challenge the view that synesthetic congruency affects temporal ventriloquism

Exposure to delayed visual feedback of the hand changes motor-sensory synchrony perception

We examined whether the brain can adapt to temporal delays between a self-initiated action and the naturalistic visual feedback of that action. During an exposure phase, participants tapped with their index finger while seeing their own hand in real time (~0 ms delay) or delayed at 40, 80, or 120 ms. Following exposure, participants were tested with a simultaneity judgment (SJ) task in which they judged whether the video of their hand was synchronous or asynchronous with respect to their finger taps. The locations of the seen and the real hand were either different (Experiment 1) or aligned (Experiment 2). In both cases, the point of subjective simultaneity (PSS) was uniformly shifted in the direction of the exposure lags while sensitivity to visual-motor asynchrony decreased with longer exposure delays. These findings demonstrate that the brain is quite flexible in adjusting the timing relation between a motor action and the otherwise naturalistic visual feedback that this action engenders

No effect of auditory–visual spatial disparity on temporal recalibration

It is known that the brain adaptively recalibrates itself to small (∼100 ms) auditory–visual (AV) temporal asynchronies so as to maintain intersensory temporal coherence. Here we explored whether spatial disparity between a sound and light affects AV temporal recalibration. Participants were exposed to a train of asynchronous AV stimulus pairs (sound-first or light-first) with sounds and lights emanating from either the same or a different location. Following a short exposure phase, participants were tested on an AV temporal order judgement (TOJ) task. Temporal recalibration manifested itself as a shift of subjective simultaneity in the direction of the adapted audiovisual lag. The shift was equally big when exposure and test stimuli were presented from the same or different locations. These results provide strong evidence for the idea that spatial co-localisation is not a necessary constraint for intersensory pairing to occur

Auditory grouping occurs prior to intersensory pairing: evidence from temporal ventriloquism

The authors examined how principles of auditory grouping relate to intersensory pairing. Two sounds that normally enhance sensitivity on a visual temporal order judgement task (i.e. temporal ventriloquism) were embedded in a sequence of flanker sounds which either had the same or different frequency (Exp. 1), rhythm (Exp. 2), or location (Exp. 3). In all experiments, we found that temporal ventriloquism only occurred when the two capture sounds differed from the flankers, demonstrating that grouping of the sounds in the auditory stream took priority over intersensory pairing. By combining principles of auditory grouping with intersensory pairing, we demonstrate that capture sounds were, counter-intuitively, more effective when their locations differed from that of the lights rather than when they came from the same position as the lights

Perception of causality and synchrony dissociate in the audiovisual bounce-inducing effect (ABE)

A sound can cause 2 visual streaming objects appear to bounce (the audiovisual bounce-inducing effect, ABE). Here we examined whether the stream/bounce percept affects perception of audiovisual synchrony. Participants saw 2 disks that either clearly streamed, clearly bounced, or were ambiguous, and heard a sound around the point of contact (POC). They reported, on each trial, whether they perceived the disks to ‘stream’ or ‘bounce’, and whether the sound was ‘synchronous’ or ‘asynchronous’ with the POC. Results showed that the optimal time of the sound to induce a bounce was before the POC (−59 msec), whereas audiovisual synchrony was maximal when the sound came after the POC (+16 msec). The range of temporal asynchronies perceived as ‘synchronous’, the temporal binding window (TBW), was wider when disks were perceived as bouncing than streaming, with no difference between ambiguous and non-ambiguous visual displays. These results demonstrate 1) that causality differs from synchrony, 2) that causality widens the TBW, and 3) that the ABE is perceptually real

The Effect of Delayed Visual Feedback on Synchrony Perception in a Tapping Task

Sensory events following a motor action are, within limits, interpreted as a causal consequence of those actions. For example, the clapping of the hands is initiated by the motor system, but subsequently visual, auditory, and tactile information is provided and processed. In the present study we examine the effect of temporal disturbances in this chain of motor-sensory events. Participants are instructed to tap a surface with their finger in synchrony with a chain of 20 sound clicks (ISI 750 ms). We examined the effect of additional visual information on this ‘tap-sound’-synchronization task. During tapping, subjects will see a video of their own tapping hand on a screen in front of them. The video can either be in synchrony with the tap (real-time recording), or can be slightly delayed (∼40–160 ms). In a control condition, no video is provided. We explore whether ‘tap-sound’ synchrony will be shifted as a function of the delayed visual feedback. Results will provide fundamental insights into how the brain preserves a causal interpretation of motor actions and their sensory consequences

The Build-Up Course of Visuo-Motor and Audio-Motor Temporal Recalibration

The sensorimotor timing is recalibrated after a brief exposure to a delayed feedback of voluntary actions (temporal recalibration effect: TRE) (Heron et al., 2009; Stetson et al., 2006; Sugano et al., 2010). We introduce a new paradigm, namely ‘synchronous tapping’ (ST) which allows us to investigate how the TRE builds up during adaptation. In each experimental trial, participants were repeatedly exposed to a constant lag (∼150 ms) between their voluntary action (pressing a mouse) and a feedback stimulus (a visual flash / an auditory click) 10 times. Immediately after that, they performed a ST task with the same stimulus as a pace signal (7 flashes / clicks). A subjective ‘no-delay condition’ (∼50 ms) served as control. The TRE manifested itself as a change in the tap-stimulus asynchrony that compensated the exposed lag (eg, after lag adaptation, the tap preceded the stimulus more than in control) and built up quickly (∼3–6 trials, ∼23–45 sec) in both the visuo- and audio-motor domain. The audio-motor TRE was bigger and built-up faster than the visuo-motor one. To conclude, the TRE is comparable between visuo- and audio-motor domain, though they are slightly different in size and build-up rate